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FOREWORD 



Toward Excellence in Secondary Vocational Education: Using Evaluation Results addresses a 
significant question of our time: "Why do so many evaluation reports remain on the shelf when 
improvement is needed in our educational system?" In this publication, the feasibility of imple- 
menting recommendations from some of the recent studies on excellence is discussed, criteria for 
estimating the appropriateness of evaluation findings are given, and recommendations are offered 
for creating an infrastructure among local, State, and Federal agencies to enhance utilization of 
evaluation findings. 

This publication is one of seven in a series produced by the Information Systems Division of 
the National Center. This series of information analysis papers is of interest to all vocational and 
adult educators, including Federal and State agency personnel, teacher educators, researchers, 
administrators, teachers, and support staff. In particular, evaluators will find the information in this 
publication stimulating and helpful in promoting utilization of evaluation results. 

The profession is indebted to Dr. Donald R. Brannon for his conceptualization of the factors 
associated with use of evaluation results. Dr. Brannon is Associate State Director for Program 
Improvement, Division of Vocational Education, North Carolina Department of Public Instruction 
He is a past president of the American Vocational Education Personnel Development Association. 

Dr. Ernest House. Professor of Educational Administration and Evaluator for the Center for 
Instruction and Curricula Evaluations. University of Illinois. Dr. Hollie Thomas. Associate Profes- 
sor of Vocational Eduction. Florida State University. Dr. N.L. McCaslin. Associate Director, and Dr. 
Steve Franchak. Senior Research Specialist, of the National Center for Research in Vocational 
Education contributed to the development of this manuscript through their reviews. The author 
extends special thanks to Dr. Rupert N. Evans. Professor Emeritus. University of Illinois. Dr. Clifton 
B. Belcher. Director of Vocational Education, North Carolina, and Dr. John Washburn. Research 
Coordinating Unit Director. Illinois, for critically analyzing earlier versions of the manuscript. Staff 
on the project included Dr. William Hull. Senior Research Specialist; Dr. Oscar Potter, Graduate 
Researcn Associate; James Belcher. Program Associate; and John Tennant. Graduate Research 
Associate. Janet Ray served as word processor operator for this manuscript. Editorial assistance 
was provided by Judy Balogh of the Editorial Services staff at the National Center. 



Robert E. Taylor 
Executive Director 
The National Center for Research 
in Vocational Education 
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EXECUTIVE SUMMARY 



The use of evaluation results often has been viewed as a means of improving vocational edu- 
cation at local, State, and Federal levels. This perception has resulted in a variety and a prolifera- 
tion of evaluation reports. Some reports have been written by evaluators distant from the scene; 
others were produced at specific sites. The increase in the use of evaluations or in the publicity 
they receive has led one prominent observer (Blaydon 1984) to remark that the growth industry in 
education appears to be evaluation reports. Unfortunately, many evaluation reports lack utility for 
easy use by administrators and others. Rarely are the persons affected by the evaluations involved 
in the design of the study. A number of recent, highly publicized evaluations of educational excel- 
lence have not addressed vocational education at all. The feasibility of implementing changes 
derived from evaluation results often is overlooked. Recent studies on educational excellence typi- 
cally identify ideal criteria, select ways to assess programs in relation to those criteria, and attempt 
to describe the discrepancy between the criteria and reality. The evaluation reports usually 
recommend that the discrepancies be decreased, without explaining how. 

This publication was written for local, State, and Federal vocational educators. Persons who 
design evaluation studies, collect evidence to support conclusions, and present results to decision 
makers need to know how to have their reports accepted by others. The basic elements of excel- 
lence and evaluation are explained in this publication by means of a three-step discrepancy analy- 
sis model. Attainment of excellence is described through the following three sets of disparate 
criteria: 

• Quantity and quality 

• Efficiency and effectiveness J 

• Equity and satisfaction 

Quantity and quality are elements of excellence, effectiveness and efficiency are often used. as 
evaluation measures, and equity and satisfaction relate to people concerns. These six criteria 
encompass a diverse and broad-based approach to the attainment of excellence through the use 
of evaluation results. These criteria should be used to judge the feasibility of strategies for imple- 
menting evaluation findings. 

National studies on secondary education are reviewed in this publication for information on 
how their recommendations would be implemented. Comments from authorities offer a less-than- 
promising picture of changes likely to occur as a result of these studies. Needed is a reliable 
means of synthesizing findings from evaluation studies and applying results to State and local 
situations. 

The search for an organizational infrastructure to bridge gaps among local, State, and 
National evaluators continues in this publication with a review of selected State and local proce- 
dures for implementing evaluation findings. The most successful procedures appear to be pro- 
grammatic in nature and interactive between the planners of evaluations and the persons affected y. 

ix 



ERIC 



7 



Variables influencing the use of evaluation results are extremely complex, Implementation of most 
evaluation findings requires modifications to accommodate unanticipated and unique needs of 
local settings. Rigid, prescriptive recommendations for change are rarely implemented 
successfully. 

Three basic organizational frameworks are presented in this paper as conceptual models for 
determining and implementing evaluation results. They are as follows: 

• Center-to-perimeter model 

• Center-node-perimeter model 

• Network model 

The center-to-perimeter model of evaluation implementation is often focused on unattainable 
ideals because the distance between where the evaluation is designed and where it must be 
implemented is too great. The center-node-perimeter model is an extension of the first form It is 
somewhat more flexible than the first but lacks the spontaneity of the network model. The network 
model allows individual units freedom to design evaluations responsive to their own needs. 
Clearly, there is a need for infrastructures to articulate and negotiate evaluation or program needs 
in vocational education at the local, State, and Federal levels. 

The prototype infrastructure presented in this paper for analyzing and using evaluation results 
is based on the following premise: persons affected by a study should be involved in the process of 
evaluation. 

This involvement may take many forms, such as helping to design the stijdy or participating in 
x a self-study prior to interaction with an external site visit team. A sense of ownership of evaluation 
results by local personnel is viewed as essential for effective implementation. The following activi- 
ties are recommended for attaining excellence through the use of evaluation results: 

• Organise local vocational excellence councils (VECs). These councils should be com- 
prised of teachers, business persons, school administrators, and parents who would be 
asked to review program improvement plans and recommendations before evaluation 
findings are implemented. 

• Upgrade access to information by using personal computers. Teachers, administrators, 
and other curriculum planners can keep up-to-date using personal computers. The com- 
puters can facilitate coordination among schools via electronic mail, newsletters, and 
other communication devices. 

• Develop coherent State improvement systems. Technical assistance in the form of inser- 
vice workshops, small grants of funds for new ideas, and other state-sponsored activities 
should encourage local innovation within predetermined, goal-oriented guidelines. 

• Agreed-upon nationwide goals for vocational education Excellence can only be achieved 
through concerted efforts from diverse constituencies within vocational education Pro- 
fessional organizations have a major catalytic role to perform in bringing about cohesive- 
ness among these groups Agreement on goals and priorities is essential for effective use 
of evaluation results. 



PURSUING EXCELLENCE WITH A DISCREPANCY MODEL 



Much ado about generic excellence per- 
vades and surrounds education, including 
vocational education. A number of studies 
about education have resulted in general 
identification of perceived ills in education 
and recommendations to cure them. How- 
ever, most of the nationally prominent 
studies have ignored recommendations for 
improving vocational education. 

This publication was written against this 
backdrop of concern for excellence in educa- 
tion and wilt help vocational educators 
determine whether or not these evaluation 
reports and their recommendations do. in 
fact, move programs toward excellence in 
vocational education. A second general pur- 
pose is to share ideas on procedures and 
organizational frameworks for better evaluat- 
ing vocational education. 

Specifically, this publication is for 
practitioners— persons at local or State levels 
who are responsible for designing and 
implementing evaluations, and those who are 
in training to do so. Furthermore, it will pro- 
mote understanding among those who are 
involved in both conducting and using the 
results from evaluations during the next 5- 
year evaluation cycle mandated by the Carl 
D. Perkins Vocational Education Act of 1984 
(U.S. 98th Congress 1984). 

There are specific requirements for'effec- 
*ively attaining and maintaining excellence in 
vocational education. These requirements 
can be met only through understanding 
essential elements of evaluation, training 
people to use correct procedures, and then 
building an infrastructure to analyze findings 
and apply relevant ones to local situations. 
An infrastructure is the framework necessary 
for the desired interrelationships among the 



parts. To meet these requirements, one must 
be able to recall and understand the 
following: 

• The elements of excellence 

• The elements of evaluation 

• Local, State, and National structures 
and infrastructures, as appropriate, 
needed, or mandated to support the 
evaluation of excellence 

• Structures and infrastructures 
needed to support the maintenance 
of levels of excellence already 
attained and to attain higher levels of 
excellence as needed 

• Specific situations during a specific 
time span in which these require- 
ments are to be applied 

• How to fit together (synthesize) the 
elements of excellence and evalua- 
tion into a practical structure 

This is a tall order, one not attainable 
within the apace limitations of this publica- 
tion Therefore, this publication discusses 
primarily the use of evaluation findings as a 
means of upgrading vocational education 
programs. Very little is said about how the 
findings are generated except for references 
to factors that impinge on use. This is an 
appropriate disclaimer because the specific 
method of collecting evaluation evidence 
should be left up to persons designing a par- 
ticular evaluation study. The purpose of the 
evaluation, the primary user audience, and 
time and cost constraints must be known to 
select the proper means of collecting data. 
This publication focuses on essential aspects 



1 



of evaluation utilization as a means of attain- 
ing excellence in vocational education. 

This publication defines and illustrates 
the basic elements of excellence and evalua- 
tion as they apply to vocational education 
and then identifies structures and infrastruc- 
tures necessary to support them. Examples 
are given to illustrate these structures among 
National studies, State evaluation proce- 
dures, and selected Federal laws pertaining 
\o vocational education. With these defini- 
tions, illustrations, and relationships as a 
background, ideas for building a system 
oriented to the needs of local vocational pro- 
grams are shared. 



The Pursuit of Excellence 

» 

Excellence in education, including voca- 
tional education, continues to be one of the 
most sought after yet elusive qualities of 
schooling. The National Commission on 
Excellence in Education's (1983) report, A 
Nation at Risk: The Imperative for Educa- 
tional Reform, asked the public to rethink its 
educational priorities. It called for reform, 
recommended an increase in high school 
graduation requirements, and asserted the 
need for higher standards without ever defin- 
ing excellence For the purposes of this pub- 
lication, excellence has been defined as ideal 
qualities and quantities of education that 
benefit students as effectively and efficiently 
as possible. 

Education is never static. Different com* 
blnatioos of educational resources exist at 
different points in time. Likewise, students' 
needs change. Older students— adults— have 
different needs than secondary students. 
Handicapped youth have different needs than 
youth learning English as a second language. 
Society delegates, through elected officials, 
the responsibility of interpreting societal 
values and setting priorities in school pro- 
grams. This occurs at all levels— local, State, 
and Federal. Optimum benefits to recipients 
of education— or educational excellence- 
will be attained whenever the highest student 



achievement occurs in relation to the re- 
sources (e.g., time, money, and expertise) 
expended. This is the concept of excellence 
used for this publication. 

It is important to keep expectations in 
perspective and to concentrate on objectives 
that are attainable. Experience helps keep 
evaluation activities on target. When one 
combines experienced management and 
leadership with evaluation activities, a new 
dimension is added to judgments about what 
is ancj what ought to be: what can be. 

This added dimension operationalizes 
excellence by putting it within reach. It helps 
us understand which levels of excellence are 
possible for different sets of circumstances. 
This operational concept is at the heart of 
being able to use evaluation results to attain 
and maintain excellence. 

The reason for this is simple. Vocational 
education operates within a condition of 
scarce resources. Those controlling sanc- 
tions and evaluations should not allow 
resources needed to maintain present levels 
of excellence to be diverted from already- 
attained areas of excellence to unattainable 
ones. A misdirection of resources could 
actually lower overall levels of excellence for 
the system. 

The idea of a pragmatic system is to 
focus on attaining and maintaining levels of 
excellence that are feasible. Determining 
such possibilities in complex reality is based 
far more on art than science and more on 
judgment than measurement. The following 
section will describe a three-stage dis- 
crepancy analysis model and address con- 
ceptual concerns about excellence and 
evaluation. 



Comparing Reality to Excellence: 
A Three-Stage Model 

Evaluation itf a process of estimating the 
worth of activities and programs for the 
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purpose of making decisions Evaluators 
strive to identify and verify vocational educa- 
tion programs that need to be maintained as 
well as those that need to attain higher levels 
of excellence. A useful tr j\ for comparing 
reality with excellence is the discrepancy 
analysis model. It helps evaluators compare 
the present state of affairs with ' what should 
be." 

The discrepancy analysts model has 
three stages. The first stage defines excel- 
lence or "what ought to be." The second 
stage identifies the discrepancy between 
"what is" and "what ought to be " The third 
stage contains an analysis of what to do 
about the discrepancy identified in the pre- 
vious stage. 

The first and foremost problem in using 
this model is that the ideal can never be 
attained. Any real entity or state of affairs 
compared to such an ideal must be judged or 
measured as lacking. Second, an ideal can- 
not be measured in units common to reality, 
although it can be described. As a result, 
surrogates, selected because they are mea- 
surable, tend to be substituted for the ideal 
People may then seek the substitutes rather 



than the ideal Furthermore, in a democratic 
society, there are varying perceptions of what 
is ideal. Consensus building is difficult. 
Methods of evaluation, therefore, should 
include ways to build consensus among 
potential users of the results. 

Excellence is composed of quality or 
quantity Evaluations cover either effective- 
ness or efficiency or some variations of 
these Ideals can have the attribute of being 
both complementary and at times contradic- 
tory, possibly offsetting each other Adher- 
ence to one ideal may diminish the other. 

The first two sets of elements— (1) qual- 
ity and quantity and (2) effectiveness and 
efficiency— are the dominant traditional ones 
and are applicable to all parts of vocational 
education The third set is more in line with 
the evolution of a democratic society in 
which vocational education functions to 
develop human resources (people) 



Stage I: Defining Excellence 

Tne first stage of a discrepancy analysis 
is modeled visually in figure 1. 




ABOExcellence; an ideal envisioning what should be. 



Figure 1. The excellence pyramid 



This model represents an ideal entity or 
state of affairs. If the tip, or apex, were to be 
reached, excellence would be attained. If 
excellence were to be attained, the focus of 
using evaluation results would then move to 
maintaining excellence. It is important to 
remember that the entire pyramid, not just 
the apex, represents excellence. Excellence 
is just as much a part of the bottom as it is of 
the top of the pyramid. In order to attain or 
maintain the top, the bottom also must be 
maintained. 

Quantity and quality. These two basic 
concepts illustrate the point that complemen- 
tary and countervailing criteria can be used 
to assess attainment of goals. Typically, as 
the demand for quantity increases, quality 
will be lower; as the demand for quality 
increases, quantity will be lower. In regard to 
vocational education, then, we must con- 
clude either that higher quality vocational 
education programs will be balanced by 
fewer programs or that higher quantity voca- 
tional programs will be balanced by lower 
quality programs. Related to the juxtaposi- 
tion of "quality" versus "quantity" are "judg- 
ment 0 versus "measurement," "words" versus 
"numbers," and "subjectivity" versus "objec- 
tivity." The evaluation of quality typically is 
done through observations that are con- 
verted into words and, therefore, is subjec- 
tive. At best, quality can be estimated. 

Quantity, on the other hand, can be mea- 
sured through numbers under the aegis of 
objectivity. Often, measurements of quantity 
are substituted for judgments of quality 
because such measurements are consistent 
with others' measurements of the same 
object taken at another time, and can be 
divorced from the evaluator. Judgments tend 
to vary more than measurements do, and 
judgments are closely aligned with the values 
of the evaluator 

Sometimes certain indicators of quantity 
are used for evaluating quality simply 
because measurements of quantity are avail- 
able Whereas the use of measurements can 



be largely objective (if assumptions are met 
and if the user follows procedures accu- 
rately), the selection of the measures is 
always subjective. Perceptions of each 
measuring device underlie judgments of its 
qualities. 

Vocational education legislation A 

major, if not the primary, goal of the Voca- 
tional Education Act of 1963 dealt with quan- 
tity Increasingly, Federal legislation has 
dealt with quality. Compare the quantity goal 
implicit in the Senate Committee on Labor 
and Public Welfare's (U.S. 94th Congress, 2d 
session 1976) summary of the "progress" of 
the Vocational Education Act of 1963 to the 
purposes of the Perkins Act of 1984: 

The Committee's hearings showed that 
the Vocational Education Act of 1963 and 
its amendments have clearly achieved 
tremendous progress. This can be seen 
in greatly increased enrollments at all 
levels, especially among post-secondary 
students and among the disadvantaged 
and handicapped, greatly increased 
expenditures of funds for vocational 
education from local, state and Federal 
sources, the construction of thousands 
of area vocational schools throughout 
the country, an increase in the number of 
trained teachers, and an expansion in the 
number and variety of course offerings 
and occupational training areas 

In contrast, the overall goal of the Per- 
kins Act of 1984 (U S 98th Congress 1984) is 
to assist the States t j expand, improve, and 
update high-quality programs of vocational 
education. According to this act, quality in 
vocational education is directly or closely 
related to— 

• the pertinence of programs to the 
workplace and to new and emerging 
technologies, 

• the responsiveness of programs to 
the current and projected occupa- 
tional needs in the State. 



4 

12 



• the capacity of programs to facilitate 
entry into and participation in voca- 
tional education and to ease the 
school-to-work and secondary-to- 
postsecondary transition, 

• the technological and educational 
quality of vocational curricula, 
equipment, and instructional mate- 
rials to enable vocational students 
and instructors to meet the chal- 
lenges of increased technological 
demands of the workplace, and 

• the capability of vocational education 
programs to meet the needs for 
general occupational skills and 
improvement of academic founda- 
tions in order to address the chang- 
ing skills needed for jobs. 

In other words, while the Perkins Act is 
in effect, all five of these indicators will be at 
the top of the excellence pyramid when qual- 
ity is evaluated based on Federal indicators 
of excellence in vocational education. Inter- 
estingly, none of these quality indicators 
refer to access to vocational programs by 
special groups or to equity. 

Th<* degree to which progress is made in 
reaching these ideals will depend not only on 
the context of each situation being evaluated, 
but also on the use of evaluation results. If 
the accomplishment of one of these ideals 
diminishes the possibility of accomplishing 
any of the others, then the indicators them- 
selves will cause negative evaluations. The 
second stage of the discrepancy analysis 
model will help to determine the relationship 
between these indicators of excellence and 
reality. | 



Stage II: Discrepancy Identification 

Evaluation using a discrepancy analysis 
model compares measurements or judg- 
ments of present reality with indicators of 
excellence, whether they fall under quality or 



quantity. This comparison of the discrepancy 
analysis is called discrepancy identification. 

Under the first of the five federally man- 
dated indicatprs of quality, an evaluation 
system would identify the degree to which 
vocational programs in reality were pertinent 
to the workplace and in new and emerging 
technologies. Hypothetical^, there is an ideal 
level at which all vocational programs would 
be absolutely pertinent, but in reality there is 
a present level of excellence already attained 
and being maintained. 

The difference between the two levels is 
a discrepancy. The size of the discrepancy 
indicates the amount of improvement in qual- 
ity and quantity necessary :o reach excel* 
lence. A reduction in the discrepancy means 
a higher level of excellence has been 
attained. The second stage is represented in 
figure 2. One should evaluate the effective- 
ness of the system. in getting to and staying 
at its highest level and its efficiency in reach- 
ing that level. 

Efficiency and effectiveness. Two con- 
cepts related to the goals and objectives of 
an organization, and prevalent to the second 
state, are efficiency and effectiveness. All too 
often it appears that there is less than ade- 
quate understanding of their meanings or 
implications for conducting and using results 
of evaluations in vocational education. First, 
we will examine effectiveness, the evaluation 
of which is highly appropriate for vocational 
education; later, under the discussion of 
stage three, we will examine efficiency, the 
utility of which is questionable in evaluating 
vocational education. 

Effectiveness traditionally has been 
defined as the degree to which prescribed 
goals or objectives are attained or intentions 
are reached. In this case, it refers to the 
degree to which excellence is attained and 
maintained. A more useful definition, how- 
ever, considers the degree to which goals 
themselves are judged to be appropriate and 
feasible for their chosen contexts 
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ABC ■ excellence; an ideal 

ADC * What is; attained levels of excellence now being maintained; reality 
ABC-ADC « Overall discrepancy 



Figure 2. Stage II: discrepancy identification 



The more distant the evaluation is from 
the actual context of the situation being evalu- 
ated, the more likely it is to stop at the 
second stage of the discrepancy analysis 
model. This characteristic is typical of most, 
if not all of the National studies on educa- 
tion. This situation may lead to unrealistic 
expectation and the identification of needs 
rather than an examination of the total pic- 
ture of what has been attained. 

The second stage of the model, discrep- 
ancy identification, draws attention to the 
difference between excellence and present 
reality The tendency is to look at the empti- 
ness of the pyramid rather than at its fullness. 
This view may backfire in focusing always on 
shortcomings with little, if any, attention 
given to present levels of excellence. This 
focus on the need for improvement may yield 
gaps in maintaining excellence. In other 
words, the model itself may affect the degree 
to which excellence is perceived to have 
been attained 



According to prevailing judgments, voca- 
tional education typically is evaluated, for 
example, by the number of graduates placed 
in occupations for which they were prepared. 
The degree to which this occurs is a measure 
of a program's effectiveness, all other things 
being equal. Unfortunately, all other things 
are not equal. For instance, the placement of 
vocational graduates in Baltimore, Maryland, 
would be more difficult than in Raleigh, 
North Carolina, due to the youth unemploy- 
ment rates for these localities. 

f 

Current legislation. The Perkins Act 
addresses effectiveness. It directs each State 
to develop measures for evaluating specifi- 
cally the effectiveness of programs assisted 
under this act in meeting the needs identified 
in the State plan. For example, States could 
use the following as measures or indicators 
of program effectiveness: 

• The occupations for which people 
train, which will reflect a realistic 
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assessment of the labor market needs 
of the State 

• The levels of skills to be achieved in 
particular occupations, which will 
reflect the hiring needs of employers 

• The basic employment competencies 
to be used in performance outcomes, 
which will reflect the hiring needs of 
employers 

• Measurements for all of these regard* 
ing their effectiveness for the 
handicapped 

In other words, these four indicators 
should be considered when each State evalu- 
ates its vocational programs based on feder- 
ally mandated indicators of effectiveness. 



Stage III: Discrepancy Analysis 

The often-neglected third stage of the 
di_screp.ajic>LanaJysis model is illustrated in 
figure 3. The third stage more closely repre- 
sents the real form of a useful evaluation if 
the intent is to make discernible progress 
toward excellence. This stage of discrepancy 
analysis takes into account the feasibility of 
implementing recommendations; the first and 
second stages of the model did not. Although 
the space ABC continues to represent our 
idealized sense of excellence, new dimen- 
sions of the model have been introduced, 
AEC. This space represents potential 
improvements— feasible changes to be 
implemented* This level of reality is different 
from the status quo, ADC, and within the 
range of possible improvements. The differ- 
ence between ABC and AEC represents a 




Unattainable 
Excellence 



Possible 
Improvements 



Reality 



ABC - Excellence; an ideal 

ADC « What is; attained levels of excellence now being maintained; reality 
AEC * What is possible 
ABC-ADC ■ Overall discrepancy 

ABC-AEC » Idealized discrepancy; unattainable excellence; political rhetoric 
A EC-ADC ■ Attainable further excellence or possible improvements 



Figure 3. Stage III: discrepancy analysis 
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level of excellence that ie unattainable. Per- 
haps the costs are too high or the particular 
attributes of excellence represented by this 
level of attainment are not valued by society. 

Previous legislation When the previous 
Federal law governing vocational education 
(Public Law 94-482, for example) is held up 
to the light of stage three, some of its ideals 
are not fully attainable. Because the field 
seemed to accept these ideals as "what could 
be'* and was evaluated using them as realistic 
standards, the field was found lacking. Some 
paraphrases suffice to describe these ideals: 

• Vocational education will be available 
to all people in all communities (not 
to some or only as feasible). 

• Vocational education will overcome 
sex bias and sex rote stereotyping 
(not reduce or help reduce). 

Additional predictors of inaccurate evalu- 
ations of effectiveness occur in situations 
where the accomplishment of one goal may 
diminish the accomplishment of others. 
Vocational Education Study: The Final 
Report (National Institute of Education 1981) 
asserts that of the purposes of Public Law 
94-482, "equalization and special needs pur- 
poses . . . are likely to be at odds with 
improvement purposes." The report also 
observes that if higher placement percent- 
ages became an indicator of excellence, the 
result would be a curtailment of enrollment; 
instructors would feel compelled to exclude 
students who may npt be easily placed after 
graduation. 

The countervailing goals of maximum 
placement and maximum access to some 
extent cancel out each other, placing voca- 
tional instructors in a perplexing position. 
Any employee who receives conflicting 
commands must be evaluated as being inef- 
fective in carrying out one or both of the 
commands. Apply this conclusion to all 
affected vocational teachers and the size of 
the problem becomes evident. 



Efficiency, on the other hand, presents a 
different story. If two vocational programs 
were equally effective, and if the -second of 
the two were to use fewer resources (inputs) 
per completer (unit of output), then the 
second program would be more efficient 
Efficiency, therefore, is the ratio of input to 
output, assuming consistent qualities for 
both. The more common term used for effi- 
ciency is productivity. The guideline for effi- 
ciency is to choose the process requiring the 
least input to produce the same product. 

The term administration in the Perkins 
Act means activities necessary for the proper 
and efficient performance of a State's duties. 
Evaluations of efficiency require measurable, 
comparable input and output. In vocational 
education, we cannot objectively isolate the 
components of input into education or output 
from education. We can isolate, describe, or 
measure the race, sex, handicap, geography, 
or socioeconomic level of students. Gradua- 
tion rates, scores on the Scholastic Aptitude 
Test (SAT), and percentages of graduating 
seniors going on to college are some popu- 
larized measurements for output. Less easy 
to relate to vocational education is employer 
satisfaction with vocational completers. 



Measurement of Goals 

Although all profit-making institutions 
have financial gain as their goal, in a pluralis- 
tic society, no similarly clear-cut goal exists 
for vocational education. Evaluating attain- 
ment of excellence in vocational education is 
complex. Taxpayers, parents, politicians, 
administrators, teachers, and students evalu- 
ate the effectiveness of vocational education 
in their own ways at different levels. Their 
views of and responsibilities for excellence 
take many directions, as do their standards of 
excellence. 

Vocational education obviously cannot 
produce one product for multiple constitu- 
ents. It must be continually involved in 
determining what its products should be, 
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including determinations by tho9e who set 
curriculum requirements, those who elect to 
attend or not attend classes, parents of 
attendees, those whose taxes pay for these 
classes, and so forth. Furthermore, there are 
multiple indicators of attainment and mainte- 
nance. Indicators are selected subjectively 
not only based upon the values of those who 
selected them, but also based on a choice of 
those indicators that are readily measurable 
and collectible. 

A hazard of selecting readily available 
measures is that just as students tend to 
study for what they think will be on a test, 
and teachers tend to teach for tests that will 
publicly reflect on their abilities, so education 
systems tend to move directly toward avail- 
able output Measures rather than toward the 
output goals these measures purport to 
represent. 

In other words, although an SAT or 
American College Test (ACT) score is a sim- 
plistic effectiveness measurement about a 
fairly circumscribed set of achievement out- 
comes of education, it tends to be substituted 
fdr the aptitude it was supposed to measure. 
Now, it is often cited as the primary measure 
of quality in secondary education, although it 
was designed only to predict grades during 
the freshman year of college. Achieving high 
average scores on the SAT seems to be the 
goal of a number of high schools instead of 
the ideal underlying accomplishments and 
student potential these scores originally were 
said to represent. Only a part of all secondary 
school students take the SAT and ACT exam- 
inations, but the scores are assumed to 
represent all students. 

From this and other evidence available, 
education as presently measured and consti- 
tuted appears to be more efficient for certain 
groups of individuals than for others. In other 
words, education apparently requires more 
or different input components for persons 
having certain measurable qualities. Efforts 
to promote increased educational efficiency 
have led to "special programs" and additional 



resources for at-risk individuals. These spe- 
cial groups need greater or different efforts 
to produce attainment levels comparable to 
those of other students. Relevant adjust* 
ments in effort need to be explored in some 
depth, especially since vocational education 
is one of those "special programs" con- 
sidered to be helpful to at-risk groups. 



Equity and Satisfaction 

In a democratic society where most peo- 
ple's basic survival needs are met, higher 
level needs become the common focus. 
Equity and satisfaction are examples of 
higher needs. All people are deemed to be 
created equal. Aligned with equity, the con- 
cept of satisfaction pervades our society. The 
current definition of management is working 
with and through people to accomplish 
organizational objectives and includes attain- 
ing an acceptable level of employee personal 
satisfaction. This approach to management 
has been evolutionary. Both equity and satis- 
faction will continue to evolve as indicators 
of quality. These two concepts also have 
spawned evaluations of how special groups 
are served and how well they perform. 

A demonstration of the degree to which 
the legislative and executive branches of the 
Federal Government have come to view voca- 
tional education excellence in terms of equity 
is evidenced in the degree to which funds 
available under the Perkins Act are targeted 
to groups requiring more efforts to achieve 
equity. About 57 percent of the basic State 
grant funds available to States and localities 
are targeted to provide vocational education 
services and activities to meet the special 
needs of selected groups: handicapped and 
disadvantaged, adults who are in need of 
training or retraining, single parents or dis- 
placed homemakers, participants in pro- 
grams designed to eliminate sex bias and 
stereotyping in vocational education, and 
criminal offenders who are serving in correc- 
tional institutions. (Note again the use of the 
absolute term "eliminate"— an ideal not 
achieveable.) 
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According to the Perkins Act (U.S. 98th 
Congress 1984), the supplemental or addi- 
tional resources and efforts needed to attain 
equity in providing vocational education 
effectively to disadvantaged and handi- 
capped students through Federal funds con- 
sist of (1) staff, (2) equipment, (3) materials, 
and (4) services not provided to others that 
are essential for disadvantaged or handi- 
capped individuals to participate in voca- 
tional education. The concept of excellence 
used especially in regard to disadvantaged 
individuals is noteworthy. The available funds 
may be used for the improvement of voca- 
tional education services or activities 
designed to provide equal access to quality 
vocational education for disadvantaged 
individuals. 

The criteria for equity and efficiency as 
goals may or may not be compatible. The 
rule of efficiency is that given two processes 
producing the same product, the one requir- 
ing the leaot input is the most efficient. For 
example, a teacher who wishes to be evalu- 
ated as efficient and who is given the oppor- 
tunity would logically choose students 
requiring less input to achieve the stated 
objectives of the course, for instance, a high 
percentage of employment. With equity as an 
evaluation criterion, students are chosen who 
require more input to achieve the objectives 
of the course, and through addition of 
resources, attempts are made to get these 
students to achieve those objectives. Equity 
in access to programs and in attainment of 
program objectives is ideal. Excess costs are 
those providing the additional resources to 
achieve equity in access and course objective 
attainment. 

In summary, equity and satisfaction are 
not only ideals of a democratic system aspir- 
ing to excellence, but equally important, they 
are needed as ways and means of attaining 
other ideals of that system, such as assisting 
the States in meeting the needs of the 
Nation's existing and future work force. 
These needs include the following: 

• Improve productivity and promote 
economic growth 



• Serve those inadequately served 

• Promote greater cooperation 

• Improve academic foundations 

• Serve the employed and unemployed 

• Serve economically o pressed areas 

• Provide supportive services, special 
programs, guidance counseling, and 
job placement 

• Increase consumer and homemaking 
effectiveness 

• Decrease sex role sterotyping 

Is it possible to attain these ideals? Is it 
possible to be evaluated as effective and effi- 
cient enough to attain the quality and quan- 
tity of programs implied in these ideals? This 
will be determined when vocational educa- 
tion is evaluated under the goals and pur- 
poses of Public Law 95-524. All of these 
ideals come directly from that law. Leaders in 
the field now have the major task of deciding 
how to meet these goals, how to influence 
evaluators to use feasiole indicators as evalu- 
ation criteria, and how to report these find- 
ings realistically. 

For the present, however, we can use the 
elements of excellence and evaluation to 
examine the degree to which vocational edu- 
cation has been realistically evaluated in the 
near past by looking at major recent National 
studies ttjat affect all of education. This 
examination is intended to give vocational 
educators interested in evaluation design a 
preview of some of the considerations they 
will have to face. The next section will first 
examine popular National studies in light of 
the elements of evaluation and excellence 
covered earlier, second, other National 
studies in general will be examined, and 
third, conclusions will be presented about 
using evaluation results, especially in voca- 
tional education. 
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STUDIES OF EXCELLENCE 



Implications for Excellence 
from National Studies 

The National studies on education serve 
as examples of evaluation efforts that typi- 
cally use ideal standards and pay minimal 
attention to feasibility. The degree to which 
they dealt in reality is indicated by the degree 
to which their study committee members 
considered the feasibility of implementing 
their recommendations. 

The National Commission on Excellence 
in Education (1983) sponsored by the U.S. 
Department of Education simply recom- 
mends that citizens across the Nation hold 
educators and elected officials responsible 
for providing the leadership necessary to 
achieve the recommended reforms. 

Peterson and the Twentieth Century 
Fund Study Task Force on Federal Elemen- 
tary and Secondary Education Policy (1983) 
do not refer to implementation planning; 
however, the Task Force on Education for 
Economic Growth (Education Commission of 
the States 1983) recommends that each State 
develop and implement a K-12 education 
improvement plan. The plan would be devel- 
oped by a broadly inclusive task force on 
education appointed by the Governor. Each 
local education agency would do likewise. 

The National Science Board's (1983) 
Commission on Precollege Education in 
Mathematics, Science, and Technology 
recommends that locf;l boards foster public 
and private partnerships for constructive 
change; that cooperative programs be estab- 
lished with the private sector for resource 
sharing, including equipment contributions; 
and that students be provided programs and 
opportunities to see science and technology 



actually applied in plants. Their teachers 
should also be employed throughout the 
entire year. 

The Carnegie Foundation for the 
Advancement of Teaching (Maeroff 1983) 
recommends that each college or university 
form a comprehensive partnership with one 
or more secondary schools and that second- 
ary schools should establish connections 
with libraries, museums, art galleries, col- 
leges, and industrial laboratories. 

The National Academy of Sciences 
(1984), as summarized in its report, High 
School and the Changing Workplace: The 
Employer's View, indicates an understanding 
that implementation of its findings will 
require dedication, patience, and very hard 
work by all concerned. 

From within the vocational education 
field, Magisos (1984) offers conclusions 
about the implications of implementing these 
National studies; he urges vocational educa- 
tors to adopt a mentality for excellence which 
extends to every course and every student. 

Magisos lists nine recommendations 
gleaned from National studies (abbreviated 
herein) for which vocational educators must 
understand implications. 

• A common core of learning 

• Stiffened requirements 

• More able teachers 

• More intensive learning 

• Meeting critical shortages of teachers 
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• Improved management 

• A strong Federal role 

• Partnerships 

• A balance between quality and equity 

Paul Peterson (1983), director of 
Governmental Studies at the Brookings Insti- 
tution and a member of the National 
Academy of Sciences 1 task force that pro* 
duced Education for Tomorrow's Jobs 
(Sherman 1983), summarizes the National 
reports as reassertions of what is well-known, 
exaggerated claims on flimsy evidence, pon- 
tification on matters about which there could 
scarcely be agreement, and recommenda- 
tions that either cost too much, cannot be 
implemented, or are too general to have any 
meaning. 

Peterson feels that such evaluations— 

• are almost certain to exaggerate the 
problem they address; 

• state only broad, gene* objectives; 

• recommend changes that are beyond 
current technology and resources; 

• do not spell out the details of their 
proposed innovations; 

• seldom call for institutional reorgani- 
zation; and 

• poo r ly document the value of the 
proposed solutions. 

Why has so little attention been given 
nationally to effectively implementing the 
results of the National studies? Peterson 
concludes the problem is that no one has 
written any reliable recipes for producing the 
desired results. 

Prakken (1984) says there is a need for 
legislators and education board members to 
learn about the values of industrial and voca- 
tional education to students ignored in the 



national statements. State and local elected 
representatives need to know about the 
accomplishments of industrial and vocational 
education. Individual shop teachers need to 
include practice in the basics and to update 
their programs in new technologies with a 
closer tie to local industries. Shop teachers 
need to acquaint school administrators, par- 
ents, the public, and students with the objec- 
tives and accomplishments of their program. 

Reliability refers to consistency. Consis- 
tent actions require consistent situations. 
The reason we do not have reliable recipes is 
not only because the environment within 
which schools function is more complex than 
described earlier, but also, as Peterson points 
out, because schools' needs are always bal- 
anced politically against other social and 
ideological concerns such as health, welfare, 
or defense. This balance is r' \ acknowl- 
edged by the National reports. Accordingly, 
Peterson concludes that the state of the 
economy will dictate the amount of needed 
resources that come to education for 
improvements far more than will any of the 
studies' recommendations. None of the 
National reports addressed education's recip- 
rocal dependence on the economy. 

Secretary of Education Bell (Fiske 1984) 
added even more succinctly to Peterson's 
summary of the possibility of attainments ■ 
from National studies by concluding that the 
number of reforms on the platter may lead to 
indigestion in application. According to the 
same report, as of September 1984, States 
have responded in the following manner: 

• Forty-two States' solutions have been 
to increase certification standards. 

• Forty have increased the number of 
academic courses required for 
graduation. 

• Thirty-two have changed curriculum 
standards required for graduation. 

• Thirty-two have changed curriculum 
standards or textbook adoption 
procedures. 
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• Twenty-four have lengthened the 
school day or year. 

According to Bell (Fiske 1984). the 
United States now faces the biggest test of 
educational leadership, administrative com- 
petence, and school diplomacy in our his- 
tory. This comes about at least partly 
because educators have been the missing 
element in making the recommendations. 
Governors and other political or business 
leaders who backed them face the challenge 
of persuading educators to carry out policies 
that largely were enacted without their partic- 
ipation and, in some cases, over their fervent 
objections. 



The Complexities of Attaining Excellence 

In recent years, there appears to be 
increased realization among national-level 
researchers of the complexities involved in 
improving vocational education and the 
necessity of focusing efforts at the local level. 
Additionally, realizations seem to be increas- 
ing that models designed to generalize evalu- 
ation results to a much larger population 
apply less effectively than more pragmatic 
ones designed to meet local needs in local 
situations. 

Some of this realization has been arrived 
at through research carried out by the 
National Center for Research in Vocational 
Education, as well as through other institu- 
tions working on a National scale. Moser 
(1983) sums up what appears to be an emerg- 0 
ing consensus at the National level, that 
replacing old materials and practices with 
new and better materials and practices is far 
more difficult for school districts than was 
once assumed. McCaslin (1974) concludes 
that the application of evaluation results may 
be worse than no evaluation activities at all if 
the local conditions, including limitations, are 
not understood by those trying to apply the 
results. 

Studies by Caplan, Morrison, and Stam- 
baugh (1975), Caplan (1980), Berman and 



McLaughlin (1975-1980), the Human Interac- 
tion Research Institute (1976), Raizen and 
Rossi (1981), Franchak and Kean (1981), 
Degner (1982). and Moser (1983) indicate 
that the dissemination of evaluation data, 
new products, new materials, or new prac- 
tices alone seldom fosters change. This kind 
of information simply does not play a central 
role in local decision making. Hull and 
Adams (1984) came to somewhat the same 
conclusion. They discovered that because 
program improvement is issue oriented and 
site specific, the worth of any product (in this 
case evaluation results) depends on its rele- 
vance to the problems at that site. 

There are complex reasons why informa- 
tion alone does not yield desired change in 
those receiving the information. The informa- 
tion can pose new risks for those in the 
organization, It can perturb the organization, 
challenge assumptions, challenge traditional 
ways of doing things along with the social 
relationships in the group, and threaten 
agendas and rationales for budget items 
(Raizen and Rossi 1981). Furthermore, new 
information ca > yield more disorganization 
as the organization to which it applies seeks 
to adapt (Youngman et al. 1980). 

If these studies are accurate, then 
designing an environment for excellence 
means designing an atmosphere that 
encourages interactive evaluations, that sup- 
ports and facilitates localities' own attempts 
to attain and maintain levels of excellence. 
The larger systems must try to design means 
to fit with the more localized ones, instead of 
the traditional attempts to mandate improve- 
ments from the top down. 



State and Local Evaluation Results 

Following are highlights of how some 
Staves and their localities have been using 
evaluation results. This information is taken 
from a June 1984 review of selected States' 
fiscal year 1982 or fiscal year 1983 account- 
ability reports in which it is mandated (by 
P.L. 94-482) that States report yearly on the 
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results of the uses of evaluations of their 
programs over the 5-year evaluation cycle. 
These examples are a sampling, not inclusive 
or representative of all the actions of all the 
States. These highlights are followed by a 
contrasting summary of evaluating State and 
local actions based on expended funds as 
presented in The Vocational Education Study 
(National Institute of Education 1981). 

Most of the selected States typically fol- 
lowed the process of ' front-end loaded" 
evaluation systems foun^ earlier in The Voca- 
tional Education Study. In these systems, 
localities typically conducted self- 
assessments by program, the State verified 
these assessments, localities developed 
improvement plans, localities may or may not 
have shared these improvement plans with 
the State, and the State may or may not have 
provided follow-up technical assistance to 
aid in making improvements. 

In North Carolina, following individual 
self-assessments, State Consultants' on site 
visits verified self-ratings of teachers. Spe- 
cific recommendations were sent by State 
consultants for each program. Local 
improvement plans were included in their 
annual applications for funding. Improve- 
ments were at local discretion. Computer 
runs of numerical ratings identified State- 
level priorities for inservice and other 
activities. 

In Maine (1983), for example, localities 
conducted a self-study using State board 
criteria. The State visited the localities to 
audit the self-study findings. Each local edu- 
cation agency (LEA) reported its findings and 
recommendations to the State board. Action 
was taken by the State board for approval or 
disapproval regarding the modification or 
continuation of the local vocational programs 
evaluated. 

Arizona (1983) reported that its assess- 
ment process included both program com- 
pliance and quality items. Areas assessed 
included planning, curriculum, staff devel- 
opment, facilities, equipment, safety, guid- 



ance and counseling, articulation, program 
development, advisory councils, placement, 
and vocational student organizations. Nine to 
twelve months after the program review, the 
team chairperson conducted a follow-up. The 
results indicated that local school districts 
actively utilized the results of the assessment 
process as a means of improving their pro- 
grams. Additionally, this process was used as 
a productive and positive public relations 
activity. 

Georgia (1983) focused heavily on moni- 
toring outcomes in programs with, for exam- 
ple, a requirement of 75 percent placement in 
postsecondary institutions and 50 percent in 
secondary schools. Georgia's position on 
using evaluation findings for results asserts 
that the ability to evaluate capability and 
adequacy is an essential part of the assess- 
ment of statewide programs of occupational 
-"education; it is clear that the effectiveness 
and value of a program can be determined 
most effectively with respect to outcomes. 

Colorado's (1983) conclusion about suc- 
cess in using evaluation results was that the 
results of the Evaluation are virtually guaran- 
teed by the process. This process included 
four types of evaluation: (1) on-site super- 
visory reviews (checklists), (2) compre- 
hensive program reviews (requested by 
localities), (3) cost-effectiveness model 
evaluations, and (4) evaluations for local 
administration and local advisory commit- 
tees. The process included 10 visits; an 
assessment of the administration, guidance 
and counseling function, and other support 
functions; a complete report with all recom- 
mendations sent to the district or institution, 
the district or institution being invited to 
respond with or without agreements; nego- 
tiated timelines for carrying out the . * 
improvements; and follow-up visits 1 year 
later* 

Colorado's cost-effectiveness model was 
used to evaluate and compare costs for full- 
time students, completion rates, placement 
rates, and the number of male, female, disad- 
vantaged, handicapped, and minorities in 
programs. It yielded a ranking, with emphasis 
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given to both the top 5 percent and bottom 
20 percent. The results of this cost- 
effectiveness evaluation system yielded the 
identification of the following areas needing 
improvement: 

• Inaccurate reporting 

• High equipment costs 

• Low percentages of placements 

• Possible poor student identification 
of vocational objectives 

• Possible teacher effectiveness 
problems 

• Errors in computing program costs 
and full-time equivalency costs 

Assistance was provided to school dis- 
tricts and institutions to rectify inadequacies. 
Colorado then went further than other typical 
States in inducing improvements: if the pro- 
gram was not improved, it was placed on 
probation. Failure to remove the program 
from probationary status resulted in the ter- 
mination of funding support under the Colo- 
rado Vocational Education Act. Thus, the 
positive results of this evaluation are virtually 
guaranteed by the process. The key parts of 
this process seem to be thoroughness, nego- 
tiated agreements, and legislative backing. 

Illinois (1984) used a variety of sources 
including the program reviews to assess 
comprehensive improvement needs and iden- 
tify priorities. From a statewide survey of 
vocational educators, on-site evaluations, 
final reports from research and development, 
individual and group concerns from such 
organizations as the Illinois Vocational Asso- 
ciation, the University Occupational Coordi- 
nating Committee, and the Illinois State 
Advisory Committee's ideas and testimony, 
improvement priorities were selected based 
on their universality, their relationships to the 
State's vocational goals, and the availability 
of resources— financial, material, human, and 
time. 



Texas (1983) reported on the following 
four purposes of validating its vocational 
education evaluation system: 

• Improving existing programs 

• Improving the design of the evalua- 
tion system 

• Providing (echnical assistance for 
program evaluation 

• Increasing local compliance prior to 
required monitoring 

Claims for the results of this system were 
conservative. For example, one out of eight 
respondents identified the lack of evidence 
that all vocational education programs had 
been evaluated prior to the local evaluation 
plan. And, one out of eight reported that eval- 
uation results were used for documentation 
purposes and to provide information to the 
Texas Education Agency. 

Connecticut (1983) reported that its 
reviews covered advisory committees, admin- 
istrative support, curriculum development, 
program of studies, public relations, safety, 
staffing, student retention, and vocational 
student organizations. Two noteworthy 
accomplishments are that the annual plans 
and proposal for ensuing years are reviewed 
to ensure inclusion of earlier evaluation 
recommendations prior to approval and that 
a review of the follow-up report prepared by 
the local education agencies evaluated show 
that better than 80 percent of the recommen- 
dations have been carried out. 

Florida (1983) has a nationally recog- 
nized program improvement system. A major 
objective of Florida is to determine optional 
methods for analyzing and disseminating 
evaluation data in order to achieve program 
improvement. 

From its yearly evaluations, Maryland 
(1983) has identified four noteworthy goals 
for improved programs: equal access, quality 
education programs, adequate support ser- 
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vices, and sound program administration. 
Equally important is Maryland's conclusion 
about continuing program strengths (areas 
where programs are most amenable to 
improvement) and chronic program weak- 
nesses (areas where program improvements 
are not so readily made). Maryland (1983) 
program strengths tended to be concentrated 
in those areas in which schools and pro- 
grams have internal control for example, 
clarification of program goals and objectives; 
utilization of varied instructional materials 
and methods; the adequacy of facilities and 
equipment. Program weaknesses were more 
apparent in areas where there were greater 
external influences, for example, placement 
services, community relations, use of com- 
munity resources, and advisory committees. 

Washington, D.C. (District of Columbia 
1983), is a city whose students' probability of 
dropping out of school is higher than that of 
graduating. According to the accountability 
report, at the end of the school year, stu- 
dents' scores on the needs assessment 
showed a marked decrease in their need for 
basic occupational information. 

According to California's (1983) report, 
program improvement funds were often used 
for routine operations rather than for pro- 
gram improvement or special programs. In 
some cases, the money was used for general 
expenses of operating or maintaining the 
educational program in the post-Proposition 
13 era. Adjustments in the accountability sys- 
tem are inevitable, as compromises have 
reduced the program improvement function. 

The Vocational Education Study 
(National Institute of Education 1981), 
funded by Congress and headed by Dr. 
Henry David, reported that in addition to find- 



ing a great variation in the uses of program 
improvement funds, there were also great 
variations among the States and localities in 
the actual use they made of the federally 
mandated program review or evaluation 
results. The study concluded that about half 
the States surveyed in 1979*80 asked local 
agencies to state in writing when and how 
they planned to correct deficiencies revealed 
by reviews. Some States simply commu- 
nicated the results and left it to local 
administrators and teachers to decide what if 
any action to take. Many of the States offered 
technical assistance in improving programs. 
The authors of the study also used funds as 
indicators of attainments in improving voca- 
tional education programs. They reported 
that the uses of Federal fi'r.ds authorized 
primarily to induce charge accounted for 
only 1.3 percent of all expenditures of Fed- 
eral funds. 

Of local education agencies' expendi- 
tures, this study found that 74 percent spent 
nothing on new programs, 70 percent spent 
less than 5 percent of their budgets on new 
services, 25 percent spent nothing on pro- 
gram improvement, and 69 percent spent less 
than 10 percent on program improvement. 

The study concluded that relatively few 
dollars were being spent locally to update 
curriculum and imprdve programs. Since the 
bulk of the education budget goes into 
instructional and administrative salaries and 
facilities, this fact is not shocking. Only a 
small part of the budget is available for pro- 
gram improvement, one of the stated pur- 
poses of Public Law 94-482. This conclusion 
should not be surprising in light of the 
emphasis placed on effectiveness in provid- 
ing the quantity of evaluation implicitly called 
for in the Vocational Education Act of 1963 
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ORGANIZING FOR EXCELLENCE 



This section introduces three 
approaches to organizing for excellence: a 
centralized approach, a partially decentral- 
ized approach, and a decentralized approach^ 
Attention is then given to program improve- 
ment structures. Organization of vocational 
excellence councils (VECs) is explored: the 
value of information networks connected by 
computers is stressed, and role responsibili- 
ties of local administrators within a State 
improvement system are discussed. 



Three Models for Excellence 

It is necessary to develop organizations 
to maintain and improve vocational educa- 
tion. Just as good organizational design will 
help people move toward the limits of needed 
quantity and quality with maximum efficiency 
and effectiveness, poor or no organizational 
design will hinder such movement. 




Figure 4. Center-to-perimQter model 
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The three organizational designs pre- 
sented in this section are intended to place 
responsibility for evaluation and excellence. 
The first model can be pictured graphically 
^^h* as a sun radiating outward. It can be thought 
of verbally as the center-to-perimeter model. 
In this model, the center has the authority, 
knowledge, and resources about improve- 
ments that simply need to be transmitted 
toward the perimeter. The job of those out- 
side the center is to accept the knowledge 
and resources and make the improvements. 
This model requires a stable, homogeneous 
environment and a simple message. In this 
model, the transmission radiates out to the 
perimeter with little if any impedance 
(although in reality there may be consider* 
able impedance). Many National commission 
reports on education assume this model 
exists. It is pictured in figure 4. 
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A second model *nay be thought of as a 
center-node-perimeter model. It does not 
have as large a center, but it has other 
smaller centers (nodes) that receive trans- 
missions, amplify them, and send them on 
toward the perimeter Other nodes may also 
be waiting to receive each transmission, 
possibly boost it. and transmit it farther. 



The center-node-perimeter model may be 
somewhat akin to the federal-state-local, the 
state-local-school, or local-school-class 
models. Any of the nodes can boost the 
transmission cledrly, distort the transmission, 
or shut down the transmission. It is pictured 
in figure 5. 




Figure 5. Center-node-perimeter model 



The third decentralized model can take a 
variety of forms. The effectiveness of this 
model depends upon each sender-receiver- 
sender coupling basically agreeing on the 
message Since individual lines of transmis- 
sion do not have to fan out so far. and 
because boosters are available at various 
levels, this system can function in a less sta- 
ble environment than the other two models. It 
can use its energy to transmit and adapt to 
diverse messages, although each message 
may have less impact than messages sent 
through the other models. Carried further, 
these ideas eventually lead to a network of 
many dimensions. See figure 6. 

In this network, the lines of communica- 
tion not only cross at every node, but can 
also bypass every node. In a sense, every 
node becomes a center and it can choose to 



communicate with every other center node 
In the network, members are not dependent 
to a significant degree on a center. Their 
behavior in identifying and making improve- 
ments is based on their self-direction and 
individual needs or wants. They are more 
concerned with learning how to solve prob- 
lems in their particular situations and with 
immediate application. Vocational education 
needs to set up a learning system that 
reflects reality and allows its members ade- 
quate independence but also provides useful 
assistance and direction. 



Characteristics of Centralization 

The proper goal of the center in develop- 
ing a viable improvement system is not to 
control but to induce members toward the 
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Figure 6. Network model 



perimeter to increase their own capabilities 
for attaining and maintaining excellence. In 
this sense, the management of individual 
local improvement situations is emphasized 
rather than the seeking of conclusions that 
are generalizable to entire populations but 
are less than applicable to individual ones. A 
number of other reforms to the system can 
be made to increase the likelihood of achiev- 
ing and maintaining excellence at the local 
level. 

Those who wish to direct program 
improvement from a center and to have dis- 
cernible and substantial progress made 
toward attaining higher degrees of excel- 
lence must send simple messages that do not 
conflict and that carry with them concurrent 
ways and means to reach the desired ends. 
These ways and means must include either 
the authority to carry out the messages or 
inducements sufficient to carry them out. 

Centralized systems are conservative; 
they work hard at maintaining the status quo. 
Such systems, therefore, can lead to more 
consistent activity across situations. They 
necessarily deal more in ideal terms and in 
generalities that may or may not apply to 
those toward the perimeter. Evaluation tends 



to be limited to determining the degree to 
which those toward the perimeter conformed 
with central ideals. Evaluation may focus on 
unattainable ideals without either acknowl- 
edging or realizing it. 

The center-node-perimeter model is an 
extension of the center-to-perimeter model, 
with each node being a center having nodes 
of its own. This model approximates the real- 
ity of the school system more closely. Units 
in this system can-receive, cut off, boost, and 
send information as needed. This model is 
less coherent overall but can be less time 
consuming because of the multiple paths 
information and resources can travel, deci- 
sions in this system tend to be more adapt- 
able to each local situation, although all 
decisions will vary somewhat from the cen- 
ter's decisions. 



Characteristics of Decentralization 

In the network, each node is a center in 
its own right. It can contact every other node 
to access that node's information at their 
mutual discretion. Decisions made at each 
node may be more tailor-made for its situa- 
tions, but coherence may be lest from an 
overall perspective. 
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The more a system becomes decentral- 
ized, the more it can satisfy the wants or 
needs of its individual units. On the other 
hand, having multiple parts in a decentralized 
system tends to lead to more duplication. 
When evaluated as an entity, this system 
appears to incur more overall waste of scarce 
human and fiscal resources, potentially lower 
quality in output, and higher costs in time 
spent by its center in purposeful 
coordination. 

Views differ on the degree to which 
decentralized systems are significant as a 
trend. Sizer (1983), author of Horace's Com- 
promise: The Dilemma of the American High 
School Today , concludes that the current 
trend toward more centralized direction of 
schools increases staff demoralization and 
ever lessens the opportunities for principals 
and teachers to adapt their programs to real 
and pressing needs. Centralized, standard- 
ized practice almost always leads to low 
standards; decentralized authority provokes 
commitment from school people on the firing 
line and increases the chance of wise 
response to the special needs and opportuni- 
ties that each student has. 

Conversely, Annison (1983), formerly of 
the Naisbitt Group that produced Mega- 
trends, is quoted in the Florida Vocational 
Journal as saying that power is now flowtftg 
back to the localities; society has said that if 
a system is big, we simply do not want it. 
Individual groups or persons do not like big 
institutions that do things for them, but insti- 
tutions that are helping groups or persons do 
things for themselves are all achieving 
success. 



Program Improvement Infrastructures 

This section will synthesize some ideas 
presented earlier, add new ones, and present 
prototype local and State improvement 
infrastructures. Many variations of the proto- 
types could be presented here. The main 



premise of this recommended prototype 
is that one requirement for acceptance of 
evaluation is to involve in the process those 
who are affected by a study and appraisal 
f ffort and those who will benefit from result- 
ihg change. 

Outside evaluations of local school sys- 
tems increasingly appear to be less feasible 
locally than helping LEAs evaluate them- 
selves. The computer is pushing organiza- 
tions toward these kinds of decentralized 
capabilities. No longer can we afford to think 4 
of program improvement only as a National 
or a State system, as has been addressed by 
most of the National dialogue. Contrary to 
the prior focus on National and State sys- 
tems, we must, according to Mills (1982), 
State director of Florida, subscribe to a com- 
prehensive approach to improving vocational 
programs at every level. All levels need to 
interact as easily as possible in program 
improvement. 

The Pennsylvania Department of Educa- 
tion's (Thornburg 1983) "agenda for excel- 
lence" states that given the diversity of the 
State and the statutory responsibilities of 
local school districts, no one model progrrm 
or series of actions should o* could be 
imposed by State government on each 
school. Rather, the State should call on local 
officials in each district to evaluate the needs 
of their students and the resources available 
and develop a strategy to ensure that a range 
of educational opportunities is available in 
each school, p 

Furthermore, we should cease looking 
upon program improvement in terms of dis- 
jointed pieces. For example, part of the 
recent past trend had been to identify and 
develop improvement products without thor- 
oughly examining how they are used. Mills 
(1982) further describes a noteworthy pro- 
cess that exhibits some worthy ideals when 
he stated that product development not 
and should not be an end in itself. The prod- 
uct must be based on a real need, be an 
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appropriate solution to that need, have a 
clearly defined target population and a well- 
developed plan for dissemination, be 
assigned the necessary resources to ensure 
that the user receives the required inservice 
training, and ensure that the total process 
from need identification through product util- 
ization be evaluated. 



Organizing Locally for Program Improvement 

Increasingly, it will become important to 
help teachers keep up-to-date with new • 
developments and to help them work 
together locally to improve programs. The 
reasons are simple. The multiple-option 
information age places a burden on the indi- 
vidual's ability to identify, collect, process, 
analyze, and use information, including evalu- 
ative information. This information overload 
will be felt not only by the local director but 
by the teacher. Annison (1983) succinctly 
summarizes the situation to the effect that in 
educational terms, these multiple options 
mean there is no way in the world, with the 
rate of growing information, that teachers 
can keep up with what is going on. 

To handle not only changing information 
about content in programs but also the mul- 
tiple evaluative information described earlier 
in this publication, organizations akin to qual- 
ity circles or task forces need to be formed 
locally. These groups should be formed 
within the traditional program areas, possibly 
by school and overall for the entire LEA. The 
functions of these peer groupings would be 
(1) program improvement, (2) public informa- 
tion, and (3) politics. 

Vocational excellence councils (VEC*). 

Organizations of teachers for achieving 
excellence can be called program improve- 
ment teams, quality circles, task forces, and 
so forth. In this publication, they are called 
vocational excellence councils. This section 
will describe an overall framework for a VEC. 
(For more ideas, read Quality Circles: Appli- 
cations in Vocational Education by Lloyd 
and Rehg [1983]. Both Lloyd and Rehg are 



professors at the Air Force Institute of Tech- 
nology, School of Systems and Logistics at 
Wright-Patterson Air Force Base, Ohio.) 

From each program area VEC, for exam- 
pie, one or more representatives would form 
the systemwide VEC. This design utilizes 
parts of the center-to-perimeter model, the 
center-node-perrmeter model, and the net- 
work model, each for reasons identified 
earlier. 

Each VEC generally would be responsi- 
ble for identifying ideals and comparing 
those ideals with present and potential situa- 
tions affecting excellence. They would use 
traditional evaluation methods described ear- 
lier, use others available, and help develop 
new ones as needed. Program area VECs 
could be organized into subgroups. These 
subgroups would be selected to provide a 
reasonable representation for individual pro- 
grams, for program clusters, or for schools in 
large program areas within especially large 
LEAs. Another organization of subgroups 
might cross LEA lines for small program 
areas in small LEAs. Program area VECs 
crossing LEA lines would, however, have 
more than one LEA center and more than 
one message coming from these centers, 
thereby causing potential confusion for the 
VECs' members. This arrangement also could 
yield conflicting evaluations of that VECs 
effectiveness by each participating LEA. 

Program-based data. Additionally, we 
must remember that each local education 
agency or postsecondary institution, at this 
time, has been through such a process and 
has collected improvement data through fed- 
erally required program reviews and the 
Office of Civil Rights (OCR) reviews. Many, if 
not nearly all, have developed program 
improvement plans from their program 
reviews, and some have developed com- 
pliance plans from the OCR reviews. Addi- 
tionally, a number of other local sets of data 
have been collected, including completer and 
employer follow-up data. Ideally, compo- 
nents of improvement plans have been devel- 
oped incorporating each set. 
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These sets of data include enrollment 
patterns that yield information on demand for 
vocational courses. The follow-up data from 
completers of vocational programs and from 
employers of these completers yield supply- 
side information about vocational programs. 
These supply-side data not only provide 
information about the outcomes of the prod- 
ucts, but can also yield information about the 
processes and content these programs use. 
The total information collected nationwide 
through the program review evaluations 
mandated for each LEA and community col- 
leges' vocational programs probably adds up 
to the most comprehensive local set of pro- 
cess evaluation data in the history of educa- 
tion, with resulting improvement plans. The 
local utilization of these data varies consider- 
ably as described earlier 

Review and release of information. The 

function of each program area VEC would be 
to review the existing improvement plans and 
the data to determine which steps toward 
excellence already have been taken, which 
have not, and why. (The State or local admin- 
istration would need to furnish these data, 
ideally in the most useful and usable manner 
possible.) Such a form would have to be 
worked out with LEAs and postsecondary 
institutions. Each program area VEC then 
plans further steps toward excellence and 
strategies for implementation and identifies 
needed resources. 

Parallel to these plans, each program 
area VEC develops public information n.ate- 
rials and strategies to share the information. 
These materials need to be developed using 
data the VEC already analyzed. The first 
choice of information would be that which 
ratifies what they are doing well and what 
they need to maintain. For example, from the 
follow-up data, each VEC could develop a 
simple comparison chart with the statewide 
youth unemployment rate, its countywide 
youth unemployment rate, and its county- 
wide vocational completer unemployment 
rate for its program area, or for vocational 
education locally overall. It could highiight 
above-average employer ratings from the 



employer follow-up (for example, over 70 
percent statewide in North Carolina on each 
of the 5 federally mandated areas: technical 
knowledge, relative training, overall training, 
attitudes, and comparable performance). It 
could highlight strengths identified through 
program reviews, weaknesses (especially 
chronic ones), steps being taken for 
improvements, and additional resources 
needed to rectify at least the chronic 
weaknesses. 

As part of its improvement plan, it would 
identify the steps to be taken on the next 
round of program evaluations mandated by 
its State legislation or the Perkins Act. It 
would seek to take all steps necessary to 
increase excellence through whatever local 
political process was available to it within 
and outside the school organization. 

This possibility would be increased 
because those who conduct vocational pro- 
grams firsthand would be involved in evaluat- 
ing and planning to achieve excellence, 
compare notes with each other individually, 
form support groups to work toward excel- 
lence in the areas in which their vested inter- 
ests lie, and receive recognition for doing so. 
Participation in a VEC and resulting 
improvements made by each participating 
teacher and by the group as a whole should 
ideally be built into a local comprehensive 
system as rungs of the career ladder model 
recommended by a number of the National 
studies, yet attempted by few. 

In summary, each of these groupings 
would be involved in organizing program 
reviews, analyzing results, developing an 
improvement plan, and implementing that 
plan. Each would also be responsible for ana- 
lyzing student and employer follow-up and 
other labor market information for implica- 
tions of maintaining and attaining excellence. 
The analysis of this information would form 
the basis for the local area improvement plan 
and feed into the LEA vocational improve- 
ment plan. Program improvement dollars 
allotted locally from Part B of Title II of the 
Perkins Act could be tied directly to the 
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development, acceptance, and accomplish- 
ment of these continuing improvement plans. 
On a purely informal basis, each of these 
groups could provide the genesis of develop- 
ing local chapters of their respective profes- 
sional organizations. 

To review the design of the center in 
each institution or LEA, each program area 
would furnish a representative(s) to serve on 
the local school system's overall vocational 
excellence council. The overall vocational 
excellence council's function would be to 
advise and assist the occupational dean, 
vocational director, or other relevant adminis- 
trator in managing the system, including syn- 
thesizing each area's program improvement 
plan into one institution or districtwide plan. 
Parallel arrangements could be made at the 
school level with each principal. 



Computer-baaed information Networks 

Teachers, administrators, and policy- 
makers have increasing access to data 
through most schools' routine tabulation of 
such statistics as average daily attendance, 
percentage of program completers' place- 
ment, and numbers of special needs students 
enrolled in programs. Easy access to indica- 
tors of performance for vocational education 
should facilitate the evaluation process. 
Computers linked to State departments of 
education can brfog listings of new publica- 
tions for vocationa^program areas (such as 
distributive education) to teachers with one 
flick of a switch. These linkages can increase 
the probability of teachers' being up-to-date 
with the latest developments in their field. It 
is the job of State-level leadership to make 
sure such data is useful, in a useable form, 
and readily accessible 

This new hardware— and software- 
promises to revolutionize data processing. 
New technologies are being coordinated into 
effective systems within each local education 
agency. These systems include data pro- 
cessing systems, office systems, telephone 



systems, and television systems. This col- 
lective instrument is called integrated 
information creation, processing, and dis- 
tribution through data, images, and voice 
(ll/CPD/DIV). 

Television systems will become interac- 
tive with increasing access to tailored data- 
bases, including those available through 
cable and satellite. The ADVOCNET system 
coordinated by the National Center for 
Research in Vocational Education is one 
major step in this direction. Telephone sys- 
tems functions will increase to include 
improved data and text handling, enhanced 
services, and data processing and voic; data- 
bases. Personal computers will make access 
to such databases easier. Office systems will 
contain word processing, electronic mail, 
electronic filing, combined copier, communi- 
cation device, and printers— and each will be 
more accessible and usable. In essence, as 
the cost goes down and power goes up, 
technology will increasingly provide equity 
among jurisdictions in access to program 
improvement data. As equity of access 
increases, program improvement capabilities 
locally will increase, and the network model 
described earlier will rise in dominance. 



Local Administrators* Roles 
and Responsibilities 

The local administrators' roles concern- 
ing VECs would be to provide leadership 
primarily through coordination and facilita- 
tion. The local vocational director Would form 
a systemwide VEC with representatives from 
each program area, as would the occupa- 
tional component for each postsecondary 
institution. Each program area representative 
would be chosen by that program area's VEC 
to be its facilitator. The systemwide VEC 
would, in turn, combine the various program 
improvement plans into a unitwide plan and 
fund each plan to the maximum extent possi- 
ble. Maximum coordination and facilitation 
skills would be needed by the local adminis- 
trator and VECs* facilitators. These funds 
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would include those available for personnel 
development, research, curriculum, exem- 
plary programs, equipment, etc., as spelled 
out in Part B of Title II of the Perkins Act. 
There, unfortunately, may lie the rub. The 
system will not work without the commitment 
of resources sufficient to improve programs 
in addition to those needed io maintain them. 

The local administrator would help 
organize the groups; find funds for them; 
sponsor inservice activities in problem- 
solving skills, evaluation, facilitation, brain- 
storming, cause and effect analysis, data 
collection methods, and so forth; and allow 
each VEC to select problems to address, to 
analyze these problems, to develop solutions, 
find to implement them; and listen to and 
support them. (See Lloyd and Rehg [1983], 
pp. 10-11.) These responsibilities call for a 
staff member's full-time efforts. In all these 
respects, the local administrator is the key to 
making the VEC system work. 

To accomplish all these tasks, the local 
vocational administrator would need to 
emphasize more working with people in 
VECs than through them. The local adminis- 
trator ideally would need to organize each 
VEC and subgroup, make sure relevant data 
are available in the form needed, make sure 
schedules are arranged to allow cooperative 
work, help provide a nonthreatening envi- 
ronment, seek funds to promote implementa- 
tion of the vocational improvement plans, 
and gain the technology to support these 
efforts. 

Additionally, local administrators would 
need to understand the local political pro- 
cess. They must try to understand bureau- 
cratic rationality, the function of information 
in their local situation, the risks such infor- 
mation can bring, and how to work with the 
VEC to develop ways to counteract behavior 
not conducive to attaining further excellence 



because of cherished local assumption and 
well-established local procedures. 



The State Improvement System 

The State must encourage the local 
improvement system and support it with 
resources. It needs to provide technical 
assistance in training people to use the sys- 
tem and to work with and through the local 
VECs to organize, analyze information, 
develop strategies, and implement programs. 
It needs to organize its next 5-year program 
review cycle around all of its databases and 
to format these data to make them useful to 
and usable for each vocational excellence 
council. The State should sponsor statewide 
or regional program improvement projects to 
address problems identified from the bottom 
up. The State also would have to provide 
maximal funding for local vocational 
improvement plans upon their acceptance 
from Part B ( Title II of the Perkins Act. 

The State would need to develop ways to 
synthesize these local vocational improve- 
ment plans and, to the degree possible, base 
State actions on these plans. It would need to 
set up model local systems to demonstrate ■ 
their workings to others. The State would 
need to train its staff in the system. It would 
need to develop the technology needed for 
the collection of local data and to develop the 
public information function as well as local 
political strategies. It would need, in other 
words, to develop a statewide vocational 
improvement plan constructed comprehen- 
sively from the bottom up instead of dis- 
jointedly from the top down, as the present 
system could be characterized. Worthington 
(1983) summarizes modernization and the 
challenges it provides as occurring on a hit- 
or-miss basis. The question is whether voca- 
tional education as a coherent organization, 
rather than a collection of isolated insti- 
tutions—can make the needed adjustments 



24 

32 



A STRATEGIC VISION FOR LOCAL, STATE, AND FEDERAL EVALUATIONS 



Increasingly, there have been realiza- 
tions at local State, and Federal levels about 
the complexity of the environment in which 
evaluations are made or not made, resources 
are allocated, and varying degrees of excel- 
lence are attained or maintained based on 
these evaluations. This section provides 
background from within and outside voca- 
tional education on perceived necessities for 
interactively attaining and maintaining excel- 
lence through evaluations at all three levels: 
local, State, and National. Some of these 
perspectives are highly generalized and ideal- 
istic; others are more pragmatic. The more 
general ones will be reviewed first. Although 
most, if not all, of these are intents of ideal 
proportions, they are needed. They do begin 
to accomplish the ideal that Michael Annison 
(1983) challenged Florida vocational educa- 
tors to achieve: that they have to start now to 
build a strategic base for what vocational 
education is going to look like over the next 
15 to 20 years. A strategic vision is the fun- 
damental concept that drives their day-to-day 
activities. Until they achieve that clear vision, 
they will not Ko ready for the future. 



Visions of an Interactive Environment 

The American Vocational Association's 
(AVA) Assembly of Delegates passed resolu- 
tion 3 on 6 December 1983 as part of its 
continuing National look at excellence in 
vocational education. One ideal of this orga- 
nization pertains to quantity and quality, and 
states simply that high-quality vocational 
education should be available to all second- 
ary students, including those with special 
needs, women, and minorities. Note the 
'should be" and "all." 



A year before this declaration, the same 
organization held hearings ("Vocational Edu- 
cation in the Comprehensive High School" 
1982) on quality in comprehensive high 
schools, from which four "prerequisites for 
remedial actions and improvements" were 
identified. These prerequisites apply to any 
level of vocational education: 

• The presence of a widely shared 
concern with the quality of the 
secondary vocational education 
enterprise and a desire to attain 
excellence 

• A structure of policies pursued at all 
three levels of government— local, 
State, and Federal— supportive of 
program and school improvement 

• The cultivation of willingness to 
apply what has been learned from 
research about such matters as— 

—the factors that meke effective 
schools and learning situations 
and 

—learning in relation to class size, 
hands-on experience, and tirr ? on 
task 

• A state of mind that does not see 
change as a threat 

The overall conclusion of the AVA report is 
that the field may be evaluation rich and 
implementation poor. Despite that, AVA pro- 
ceeds to note that there is little mystery 
about what needs to be done and what, in 
fact, can be done. The problems lie in mus- 
tering resources, energy, and will for the 
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doing. On this score, the point is made that a 
powerful stimulus for redirection of voca- 
tional education in the high school, not yet 
adequately used, lies in making periodic 
assessments of every aspect of the enterprise 
that bears upon performance, ranging from 
local and State planning through availability 
of resources and their use, to the competen- 
cies acquired by students, to classroom 
procedures and methods of instruction. 

The National Association of State Direc- 
tors of Vocational Education also went on 
record in December 19B3 as identifying goals 
for interactive program improvement: 

• Comprehensive planning in program 
improvement 

• Coordination of program improve- 
ment processes within and among 
levels 

• Implementation of program 
improvement outcomes 

• Evaluation of program improvement 
activities 

• Involvement and active participation 
of those affected 

• Cooperation with other groups con- 
cerned with education and work 

This organization further recommended 
impaneling a National vocational-technical 
education program improvement coordinat- 
ing council for recommending "coordinative 
strategies'' to resolve problems among the 
respective organizations and agencies 
involved in program improvement. Corre- 
spondingly, it recommended that each State 
director lead the States to— 

• assess State program improvement 
needs, 

• recommend statewide program 
improvement priorities, 



• coordinate State program improve- 
ment activities, and 

• articulate with other programs 
through the National council. 

The council's functions would be to take 
a leadership role in developing a systematic 
process to identify (from the States) National 
priorities for program improvement; deter- 
mine what States are doing in these priority 
areas; and, via electronic means, demon- 
strate how States are addressing current and 
future priority areas. 

These National positions are not new. 
They have evolved during the last few years. 
A series of actions preceded them. The fol- 
lowing are examples. Again, these are not to 
be inclusive or representative of all such 
actions. 



Some Preliminary Actions 

In 1980, the report from a 1979 federally 
sponsored seminar on research coordinating 
units in vocational education addressed the 
theme of breaking with the less-than- 
comprehensive focus: 

• Program improvement ought to be 
viewed as a comprehensive research, 
development, and diffusion process 
rather than as a collection of 
functions. 

• The continual development and 
training of the personnel involved in 
vocational education should be an 
integral part of program improve- 
ment. (Miller 1980) 

Two years later, during the 1981 Ameri- 
can Vocational Association (AVA) national 
convention in New Orleans and at the behest 
of the American Vocational Education Per- 
sonnel Development Association (AVEPDA), 
the heads of each of the following organiza- 
tions met to discuss their respective needs 
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for comprehensive program improvement: 
National Research Coordinating Unit Associ- 
ation, National Association of State Directors 
of Vocational Education, American Voca- 
tional Education Research Association 
(AVERA), Vocational Instructional Materials 
Association, New and Related Services Divi- 
sion of AVA, National Council of Vocational 
Teacher Educators, and National Network for 
Curriculum Coordination in Vocational- 
Technical Education (NNCCVTE). 

Following this meeting, at the same con- 
vention, AVEPDA and AVERA sponsored a 
joint session entitled "Fitting the Pieces of 
Program Improvement Together/' Following 
are selected key points, excerpted from a 
tape of this joint session, about building a 
collaborative National environment condu- 
cive at all levels and across related functions 
for achieving and maintaining excellence 
from a variety of perspectives. 

Dr. Rebecca Douglass (National Network 
for Curriculum Coordination in Vocational- 
Technical Education) said: 

At the Federal level, State curriculum 
needs assessments should be synthe- 
sized to develop National priorities and 
screen priorities so that they not be 
duplicated at the State and National 
levels. A panel of leadership from each of 
the professional groups should be se- 
questered to do this each year. Curricu- 
lum activities at the Federal level would 
be targeted to major design, formatting, 
delivery, management, and evaluation 
concerns. The development would be left 
to the States with coordination through 
NNCCVTE and the National Center for 
Research in Vocational Education. 

According to Dr. Robert E. Taylor 
(National Center for Research in Vocational 
Education): 

We have not had. nationally, general 
agreement on priorities. There has been 
a pattern of change every few years; 



there has not been the durability of 
attention to each problem to go the full 
range of the processes that could be 
applied to the problems; we need to do a 
better job nationally of tying resources to 
goals. We need to improve the quality 
and quantity of interactions. We need to 
pursue additional dialogues based on 
our loose couplings and reciprocal 
benefits. 

Dr. Jerome Moss (Department of Voca- 
tional Education, University of Minnesota) 
stated: 

We need to differentiate between reform- 
oriented program improvement and 
renewal program improvement. Reform- 
oriented inquiry is best done by those 
not just in the vocational system, but 
managed by those concerned with all 
employment-related training. The re- 
newal inquiry is best managed by opera- 
tors within thfe vocational education 
system. Those closer to the bureaucracy 
are better for fine-tuning the system. The 
Federal Government should maintain a 
mixed portfolio with most of its focus on 
renewal, some on reform. 

Dr. Henry David s (The Vocatiqml Edu- 
cation Study) comment was: 

The universal threat is the struggle 
between the parts and the whole. We 
shouldn't worry so much about networks 
and interfaces as cultivating the capacity 
for improvements. This requires settling 
for more than what we have had. not 
less. The Federal funding mechanisms 
are not compatible with (Federal) goals. 
The business of improvement begins 
with individuals . . . and we have no 
reward system for what we call "self- 
learning." The nice thing about models is 
that there is always more than one. And 
thv 'eduction to a singular model would 
not be desirable, because it would have 
to be based on the lowest common 
denominator. 
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After a number of dialogues, Joe D. Mills 
(1982), State director of vocational education 
in Florida and then president concurrently 
both of the AVA and the National Association 
of State Directors of Vocational Education, 
formally articulated the evolution of ideas 
about interactive program improvement 
across the National environment. Mills stated 
that program improvement implies change, 
and educators should not and cannot expect 
to improve vocational programs without con- 
sidering possible changes in any and all 
facets of the total educational delivery sys- 
tem. It is of little value to produce improved 
instructional materials if their ultimate use in 
the classroom is dependent on a dissemina- 
tion system that cannot deliver the product. It 
is not reasonable to expect utilization of the 
product if its use is dependent upon an inser- 
vice training program delivery by untrained 
trainers. 

Mills* (1982) conclusion on the holistic 
approach needed through the entire National 
environment reflected the views of many 
others, emphasizing the need to subscribe to 
a comprehensive approach to improving 
vocational programs at every level. Program 
improvement should be looked at in terms of 
a total activity or process— not just in terms 
of its component parts such as those spelled 
out in past legislation. 



Focus on Implementation 

It is crucial to spend at least as much 
money, time, and energy in examining, 
researcning, evaluating, or developing 
methods, and so on, to determine how to 
attain improvements as is spent on evalua- 
tions to determine which improvements are 
needed. It is not uncommon to hear persons 
experienced in local vocational education 
administration or teaching conclude that 
their programs would be vastly improved if 
they knew how to make or could make the 
improvements they already know are needed. 
As the farmer said, "I already know how to 
farm better than I actually do." 



As much time should be spent in 
research or curriculum development to 
determine how findings can be used and then 
to put them into effect. And at least as much 
time should be spent on implementing the 
critical activities and steps necessary to put 
the findings into effect as was put into devel- 
oping them. In other words, quality must be 
emphasized in program improvement activi- 
ties over quantity. This ideal emphasis is not 
an easy one in an environment supported by 
minimum tax bases. The assumptions and 
goals that underlie the center-to-perimeter 
model as it applies to the Perkins Act should 
be debated nationally and tested, if possible, 
to determine if and when they are feasible. 

The ideal of research— generalization of 
findings— is not the main ideal for a compre- 
hensive program improvement system. Pro- 
gram improvement is dependent on each 
situation. The philosophical visions that 
should drive program improvement are 
pragmatism and existentialism. What must be 
designed are sys*<?ms to maximize the capa- 
bilities, including incentives, for situational or 
site-specific program evaluation and 
improvements at local and State levels. 

Pennsylvania's conclusions about situa- 
tionally specific improvements reflect similar 
concerns. A report entitled Turning the Tide: 
An Agenda for Excellence in Pennsylvania 
Public School (Thornburg 1983) concluded 
that the schools can adapt a variety of addi- 
tional strategies to increase the quality of 
education provided to their students. Also, as 
noted in High School: A Report on Second- 
ary Education in America (Boyer 1983). pub- 
lished by the Carnegie Foundation for the 
Advancement of Teaching, strategies to 
improve public education will differ from one 
school to another. 

Upon close examination, one finds that a 
number of Federal perspectives have evolved 
in support of the aforementioned interactive 
system. The discrepancy between perspec- 
tives and action may largely be due to politi- 
cal realities. In the name of flexibility, for 
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example, managers of vocational State and 
local bureaucracies have been allowed to 
decide whether funding from P L. 94-482 for 
excellence meant funding more program 
improvement activities or funding more per- 
sonnel. Bureaucratic rationality answered the 
question for them\vith the obvious choice of 
more personnel. Bureaucracies are evaluated 
more on quantity than quality. Furthermore, 
in trying to be of maximum benefit to at-risk 
groups and to the labor market, the Federal 
Government, although well-intentioned, sent 
very complicated and somewhat conflicting 
messages regarding the operationalization of 
P L. 94-482 through a center-to-perimeter 
model demanding simple messages. 

To ensure that more action is addressed 
to improving programs, either some flexibility 
must be sacrificed to gain correspondence 
between Federal purposes and funds, or the 
subsequent Federal evaluations will have to 
address the ideals rather than actually 
intended accomplishments. Furthermore, 
ensuring that evaluations of effectiveness 
take place should take into account the 



impact of counteracting goals on attainments 
and resulting evaluations. In essence, actions 
must be evaluated in relation to realistic con- 
texts, processes, products, and outcomes. 

Local, State, and Federal evaluations 
ideally must base assumptions and expecta- 
tions more on comparisons with the possible 
than on political rhetoric. Only by consider- 
ing the art of the possible will the accuracy of 
evaluation results be increased and the sys- 
tem helped to produce the results that are 
substantially closer to excellence. 

Only when the art ol the possible is 
included consistently in the evaluation of 
vocational education will the probability of / 
excellence increase. As the National 
Academy of Sciences study headed by 
Blaydon (Sherman 1983) stated, "we would 
like to see vocational education become an 
equal partner with college preparatory edu- 
cation in the education system as a whole. 
The most effective vocational programs are 
deserving of that respect now, and we would 
like to see all programs raised to that level of 
quality and esteem" (p. 93). 
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